maybe_ascii: fold š/č/ş/ș/ț/ĉ and Cyrillic щ/ц to digraphs - #293

Fixes #292.

maybe_ascii folded š/č/ş to bare s/c, but name spellings in the wild mostly use sh/ch. The corpus measurement in #292 shows a net gain of +5,214 agreeing spellings for the six Latin characters.

Change

• Latin digraph fold: š/ş/ș → sh, č/ĉ → ch, ț → ts. This runs after the per-script pass and before the NFKD mark strip, which otherwise removes the diacritic. The input is NFC-composed first, so decomposed input (s + U+030C) is covered too.
• Cyrillic pre-pass: щ → shch, ц → ts, substituted before the ICU4X Cyrillic transliterator. The transliterator's output for these (ŝ, c) can't be told apart from other letters afterwards.
• ž and ţ keep folding to the bare letter: for ž the corpus is a coin flip, and for ţ the current fold is better.
• Uppercase maps to title case (Š → Sh), following the existing Þ → Th.

Metaphone runs on the ASCII form, so its codes now agree with the common spellings (Kašerininov and Kasherininov both give KXRNNF).

Downstream impact

This changes the output of maybe_ascii, and therefore NamePart.ascii, comparable and metaphone, plus address tokens (шоссе → shosse, which forms.yml already lists). Anything that stores folded forms, such as yente indexes, should be rebuilt after upgrading. This needs a changelog note and a version bump.

Not covered here: х → h (vs kh) and й → j (vs i/y).

Tests

• cargo test: 273 passed
• pytest: 548 passed
• mypy --strict: clean
• clippy (with and without --features python): clean
• cargo fmt --check: clean

🤖 Generated with Claude Code